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An Impact Evaluation of the Blended Core Mathematics Program for 
Elementary Grades 


Introduction 


Founded in 1969, Curriculum Associates provides a variety of educational products and 
services with the goal of improving education for students and teachers. Three Curriculum 
Associates products include i-Ready® Diagnostic (available for K-12), i-Ready® Instruction 
(available for K-8), and Ready® Mathematics Core Curriculum (available for K-8). The i-Ready 
Diagnostic assessments (a) are online, computer-adaptive assessment that pinpoint student 
needs at the sub-skill level and (b) help monitor the extent to which students are on track to 
achieve end-of-year targets. The i-Ready® Instruction provides online, individualized instruction 
for students and is designed for use with the /-Ready Diagnostic. Ready Mathematics is a 
curriculum program that can be used as a core curriculum or to enhance mathematics 
instruction. It is based on the Common Core State Standards (CCSS) that offers rigorous 
instruction and practice for students and resources and tools for teachers. 


Ready Mathematics is a rigorous, yet reachable mathematics program that, when layered with 
i-Ready Diagnostic and i-Ready Instruction, offers a fully integrated blended learning program. 
When /-Ready Diagnostic, Instruction, and Ready Mathematics are used together as a core 
curriculum, they are referred to as a Blended Core Mathematics Implementation. This innovative 
solution was built from scratch by a print and digital development team to align to the standards 
and to work together seamlessly. When Ready Mathematics is implemented as a core 
curriculum with /-Ready as a complementary digital program, teachers get robust data to guide 
their instruction, while each student receives a personalized /-Ready Instruction path to 
complement the instruction and practice in Ready Mathematics. 


The Human Resources Research Organization (HumRRO) conducted an evaluation to examine 
the impact of Blended Core Mathematics on mathematics achievement for students in grades K-5 
compared to use of i-Ready Diagnostic only. This study was designed to meet the required rigor 
of the What Works Clearinghouse (WWC) 4.0 standards to achieve a rating of Meets WWC Group 
Design Standards with Reservations (WWC, 2017a), and to meet guidelines for a Level 2 (or 
Moderate) rating for the Every Student Succeeds Act (ESSA) guidance for evidence-based 
research (U.S. Department of Education, 2016). This was achieved by using quasi-experimental 
design (QED), establishing baseline equivalence between the treatment and comparison groups, 
using an outcome measure acceptable by WWC, including baseline achievement as a covariate, 
and sampling design that mitigates the effects of any confounding factors. 


Defining Blended Core Mathematics 


Curriculum Associates has developed a Blended Core Mathematics Logic Model highlighting 
the key resources that make up Blended Core Mathematics, the strategies and activities 
important for implementation, the expected outputs, the intended outcomes, and the intended 
impacts. The strategies and activities describe actions by teachers recommended to obtain the 
long-term outcome of improved student learning in mathematics. Among others, these include 
the use of i-Ready Diagnostic data to determine students’ learning needs, use of Ready 
mathematics in class every day as the core curriculum, and use of i-Ready Instruction for 
student support outside of whole-class instruction. Curriculum Associates believes assessment 
results (i.e, -Ready Diagnostic) should inform targeted student instruction and intervention. For 


An Impact Evaluation of the Blended Core Mathematics Implementation for Elementary Grades 1 


PB HUMRRO 


the full suite of Blended Core Mathematics, we can consider the Diagnostic as the monitoring 
tool, and Ready curriculum and /-Ready Instruction as the intervention piece. The Blended Core 
Mathematics Logic Model is provided in Appendix A. 


In addition to the Blended Core Mathematics logic model, Curriculum Associates provides 
guidance to districts and schools on how to implement /-Ready Instruction to best benefit 
student learning (Curriculum Associates, 2019). Guidance indicates students receive greater 
gains with at least 30 — 49 minutes of i-Ready Instruction use for each subject area. In addition, 
Curriculum Associates recommends use for at least 18 weeks between a fall i-Ready Diagnostic 
administration and a spring administration (Curriculum Associates, 2018). 


Though Curriculum Associates intends some flexibility in Ready and i-Ready Instruction use, the 
logic model and guidance documents provide direction for implementing Ready and /i-Ready 
Instruction with fidelity. This information was used to help identify schools and students eligible 
for participation in this Blended Core Mathematics evaluation. 


Research Questions and Study Design 


The primary purpose of this evaluation was to estimate the impact of using Blended Core 
Mathematics on student achievement. Particularly, we were interested in how the use of the 
three products that make up Blended Core Mathematics impact student achievement in 
mathematics beyond the use of i-Ready Diagnostic only for mathematics. Our research was 
focused on one primary research question, addressed separately for each elementary grade 
from kindergarten to grade 5. 


e What is the impact of Blended Core Mathematics on student achievement in 
mathematics compared to the use of i-Ready Diagnostic alone? 


Our hypothesis was student mathematics achievement would improve at schools using Blended 
Core Mathematics over use of i-Ready Diagnostic only. This hypothesis was based on the belief 
that students benefit from the /-Ready Instruction targeted to their specific needs, and from 
Ready Mathematics as their core curriculum. We predicted that the use of these products would 
benefit student achievement in mathematics and would be reflected in i-Ready Diagnostic 
scores in the spring following one school-year of use. 


Cluster-Level Design 


The unit of assignment for this study was the school, because assignment into Blended Core 
Mathematics (treatment) or i-Ready Diagnostic Only (comparison) group occurred at the school 
level. The unit of observation in this study was the student, with student-level achievement on 
the i-Ready Diagnostic serving as the baseline measure and the outcome measure. 


Outcome Measure 


The mathematics /-Ready Diagnostic assessment was designed to be aligned to today’s 
college- and career-ready standards and to provide results that inform student placement 
decisions, offer explicit instructional advice, and prescribe resources for targeted instruction and 
intervention. The /-Ready Diagnostic is currently used by more than 6.5 million students across 
the United States — and users sometimes incorporate other /-Ready products (i.e., Instruction, 
Teacher Resources), though this is not a requirement. 
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To provide evidence the i-Ready Diagnostic measures skills consistent with student 
expectations, Curriculum Associates has conducted multiple linking studies to examine the 
correlation of i-Ready Diagnostic scores with scores from national and state summative tests for 
mathematics at grades 3 — 8. Linking studies using 2016 data examined the correlation between 
i-Ready Diagnostic and the Smarter Balanced summative assessments, the Partnership for 
Assessment of Readiness for College and Careers (PARCC), and multiple state testing 
programs (North Carolina, New York, Tennessee, Ohio, Mississippi, Michigan, Indiana, Florida, 
and Georgia). These studies show strong correlations between i-Ready Diagnostic scores and 
scores on these national and state tests. The average correlations across grades between the /- 
Ready Diagnostic for mathematics and the national and state mathematics assessments ranged 
from 0.82 (North Carolina End-of-Grade assessments) to 0.88 (Smarter Balanced and Michigan 
M-STEP). These studies provide evidence that the -Ready Diagnostic content is highly 
consistent with what students across the United States are expected to learn (Curriculum 
Associates, 2019). Curriculum Associates has also recently completed linking studies for 
Colorado, Kentucky, and Missouri. In addition, Curriculum Associates has commissioned Odell 
Education and others to complete alignment studies to demonstrate the degree of alignment 
between the content on i-Ready Diagnostic and current sets of state standards. Specifically, 
they have conducted alignment studies for the Common Core State Standards (CCSS), and for 
the Louisiana, Indiana, Ohio, Michigan, Florida, and South Carolina state standards. 


Curriculum Associates released /-Ready Diagnostic in the summer of 2011. Since then, /-Ready 
has been reviewed and approved at the state level as an assessment, instructional resource, or 
intervention in Arizona, California, Colorado, Connecticut, Delaware, Florida, Georgia, Idaho, 
Indiana, Massachusetts, Mississippi, Nevada, New Mexico, New York, North Carolina, Ohio, 
Oklahoma, Oregon, Tennessee, Utah, and Virginia. 


i-Ready Diagnostic for mathematics measures achievement aligned to common content and skills 
with demonstrated test score reliability. Marginal reliability ranges from .92 to .96 and test-retest 
reliabilities range from .71 to .86 for mathematics in kindergarten through grade 5. Therefore, this 
assessment meets the WWC 4.0 standards for an acceptable outcome measure (WWC, 2017a). 
The i-Ready Diagnostic was used as the baseline and outcome measure for all students 
participating in this study (i.e., -Ready Instruction students and comparison group students). 


The i-Ready Diagnostic is intended to be administered in a standardized manner across schools 
(Curriculum Associates, 2019b). Specifically, at grades K-5 teachers are to schedule the first (fall) 
Diagnostic 2 — 3 weeks into the school year in two 45- to 50-minute sessions. They are to test 
technology to ensure proper function and have pencils and paper available as scratch paper. Test 
administrators are to provide instructions to their students and motivate them to do their best. As 
students are testing, teachers are to monitor students. 


Identifying Schools and a Baseline 
Treatment Schools 


Curriculum Associates defines Blended Core Mathematics as contemporaneous use of three 
key components: (a) Ready Mathematics as their core curriculum, (b) i-Ready Diagnostic, and 
(c) i-Ready Instruction. A primary goal for this study was to include schools that implemented 
Blended Core Mathematics with fidelity. Prior to identifying a sample of schools, HumRRO and 
Curriculum Associates worked collaboratively to establish the definition of an adequate Blended 
Core Mathematics implementation. This definition was based primarily on the directions given 
by Curriculum Associates to districts and schools for implementing Blended Core Mathematics. 


An Impact Evaluation of the Blended Core Mathematics Implementation for Elementary Grades 3 


PB HUMRRO 


For a school to be considered as implementing Blended Core Mathematics with adequate 
fidelity it needed to meet the following eligibility criteria: 


e Administer the i-Ready Diagnostic for mathematics a minimum of two times during the 
school year — fall and spring — to most students. Schools using the i-Ready Diagnostic 
for students requiring extra support only were eliminated from inclusion. Information on 
this eligibility criterion was obtained through review of i-Ready data. 


e Show solid use of i-Ready Instruction. Schools had general education students engaging 
with i-Ready Instruction, with at least some engaging for an average of at least 30 
minutes per week for a minimum of 18 weeks between the fall and spring i-Ready 
Diagnostic administrations. Information on this eligibility criterion was obtained through 
review of i-Ready data., and discussions with Curriculum Associate staff who worked 
closely with the Blended Core Mathematics districts. Consistent with the /-Ready 
Diagnostic criterion, those using i-Ready Instruction only for students requiring extra 
support were eliminated. 


e Schools from districts that had adopted Ready Mathematics as approved curriculum and 
selected it as their primary, core mathematics curriculum. Eligibility was determined 
through conversations with Curriculum Associates staff who had worked closely with the 
Blended Core Mathematics districts and schools by providing professional development 
and ongoing leadership. We eliminated those schools and districts that had not yet 
established Ready Mathematics as their primary curriculum. 


Our study sought to include recent data, thus we used only schools that began Blended Core 
Mathematics implementation during the 2016-17 school year and continued use during the 
2017-18 school year. To maximize power, we sought to include all schools that met our 
definition of Blended Core Mathematics use. We began with a list of ten school districts 
identified as potential Blended Core Mathematics users based on purchase information. 
Through discussions with the Curriculum Associates staff working directly with the schools in 
these districts, we learned that all schools in two of the districts were not using Ready 
Mathematics as their core curriculum. These districts were eliminated. In the 8 remaining 
districts there were 37 schools with i-Ready Diagnostic and Instruction data. These schools 
included various configurations of grade-levels. HumRRO’s review of /-Ready Instruction data 
indicated two middle schools used /-Ready Instruction for very few students - Curriculum 
Associates staff confirmed that these schools used /i-Ready Instruction only for students 
identified through response to intervention (Rtl). This left 35 Blended Core Mathematics schools 
that met our definition of full Blended Core Mathematics implementers. 


Identifying the Baseline 


Curriculum Associates research staff identified elementary schools across the United States 
that purchased the Blended Core Mathematics program. HumRRO and Curriculum Associates 
research staff then held multiple discussions with Curriculum Associates professional staff who 
worked closely with these schools, providing professional development, conducting site visits, 
and providing ongoing leadership support. Through these conversations we learned that 
schools were typically asked to concentrate on implementing Ready Mathematics first, the /- 
Ready Diagnostic administrations second, and add in /-Ready Instruction as a final priority. 
Curriculum Associates staff noted there is a learning curve for implementing the new Ready 
curriculum; it tends to take time for educators to become familiar with the Ready Mathematics 
teaching practices. Curriculum Associates staff also noted that at some schools, educators used 
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familiar materials and lessons instead of Ready Mathematics curriculum while they were 
familiarizing themselves with the new curriculum. 


We also inquired about /-Ready Diagnostic and Instruction use. Because using the /-Ready 
Diagnostic is less time consuming than /-Ready Instruction, Curriculum Associates professional 
staff noted many schools start using the diagnostic early in the adoption process for Blended 
Core Mathematics. i-Ready Instruction, however, is often the last focus, incorporated once other 
pieces of the program are in place. School leaders and educators need to strategize for how 
they can carve out time for ij-Ready Instruction and ensure the necessary technology is 
available. At some schools, it took a semester or more to have the resources in place to 
implement /-Ready Instruction. 


Using Curriculum Associates’ internal i-Ready data, HumRRO examined /i-Ready Diagnostic 
and /nstruction use over time for the schools that met the eligibility criteria for inclusion in the 
treatment group. We found, consistent with what we learned through discussions, the number of 
students using /-Ready Diagnostic and Instruction increased for the Blended Core Mathematics 
schools between the first and second year of having purchased the Blended Core Mathematics 
program. These data suggest full implementation of Blended Core Mathematics did not 
occurring during the first year. Based on the information collected, we determined the starting 
point for full implementation of Blended Core Mathematics was the start of the school year 
following the purchase of the Blended Core Mathematics program. In other words, because all 
schools identified for our study purchased Blended Core Mathematics for the 2016-17 school- 
year, our baseline was identified as the start of the 2017-18 school year. 


Comparison Schools 


Once treatment schools were identified and the baseline determined, we identified a set of 
potential comparison schools. We examined 2016-17 and 2017-18 /-Ready data to identify a 
set of schools with grade K—5students for whom (a) /-Ready Instruction was not used by any 
student both years, and (b) the mathematics i-Ready Diagnostic was administered to a 
minimum of 40 students for both the fall and spring administration for the 2017-18 school year. 


Analysis 
Power Analysis 


Power analyses were conducted to identify the number of schools required in the treatment and 
comparison group at each grade level for sufficient power to reject the null hypothesis that there 
is a true difference in student mathematics achievement between the treatment and comparison 
group. Statistical power is influenced by various factors. We considered the relationship 
between the baseline and outcome variable, typical number of students in our eligible treatment 
and comparison schools, and estimated intraclass correlation coefficients (ICCs) based ona 
previous /-Ready impact study to identify the number of schools needed at each grade level to 
reach a power level of 0.80. A 0.80 power level provides an 80% chance of detecting a 
significant difference with 95% confidence. 


Our power analysis suggested having greater than 40 schools for each analysis would be ideal 
for determining an impact. Though we had 32 total treatment schools with at least one grade in 
K—5, not all schools included use of Blended Core Mathematics at each grade level. Similarly, 
our potential comparison group of schools had various grade compositions and did not always 
use /-Ready Diagnostic at every grade. However, we had a much larger pool of potential 
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comparison groups from which to select. Based on our power analysis, we determined to match 
two comparison schools to every one treatment school to increase power. 


Achieving Baseline Equivalence 


Once schools meeting the eligibility criteria for the treatment and comparison group were 
identified, matching was conducted to select comparable groups of schools and students. First, 
matching was conducted at the school level to ensure key school demographic characteristics 
were similar between the groups of treatment schools and comparison schools. Schools were 
matched on the following variables: 


Percent students eligible for free or reduced lunch (FRL) 
Percent students limited English proficient (LEP) 
Percent students with disabilities (SWD) 

Percent students white 


These variables were selected as they are known to be related to student achievement, and 
reliable data are available for public schools across the country, including all schools meeting 
the criteria for our sample. The Department of Education Civil Rights Data Collection Website 
was used to obtain the most recent publicly available school-level information, from the 2015-16 
school year, on percent white, percent limited English proficiency (LEP), and percent students 
with disabilities (SWD). The Common Core of Data’, also provided by the Department of 
Education through the National Center for Education Statistics, was used to obtain school-level 
information on the percent of students eligible for the free and reduced lunch program (FRL) 
and to ensure only public institutions were included in the comparison group, to be consistent 
with the school type of the treatment group. 


Logistic regression was used to compute a propensity score for each school in the treatment 
group and the comparison pool. The model predicted the chance that each school belonged to 
the treatment group through a propensity score between 0 and 1. In order to increase the 
number of schools at each grade level, two comparison schools were matched to each 
treatment school. Using a nearest neighbor matching approach, baseline equivalence was met 
on the sample characteristics with no effect size larger than 0.50 (WWC, 2015). We began by 
matching all initially identified 35 treatment schools to two comparison schools. This resulted in 
35 treatment schools and 70 matched comparison schools. To account for our focus on 
elementary schools (K-5), we removed the three treatment schools and 11 comparison schools 
for which no grades K-5 were present, resulting in 32 treatment and 59 comparison schools with 
at least one grade K-5. We found baseline equivalence was maintained for this reduced set of 
schools. Table 1 provides descriptive statistics for each matching variable for the elementary 
schools in the Blended Core Mathematics (treatment) group and /-Ready Diagnostic only 
(comparison) elementary group, as well as the effect sizes of the difference between the means. 


1 https:/Avww2.ed.gov/about/offices/list/ocr/data.html 
? https://nces.ed.gov/ccd/ 
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Table 1. School Demographic Variables and Effect Size Differences Between the Blended 
Core Mathematics (Treatment) and i-Ready Only (Comparison) Group. 


Blended Core Mathematics i-Ready Diagnostic only | Hedge’s g | 


WEVar-Ve) (=) (NEY) (N = 59) Effect 
Mean (SD) Mean (SD) Size 

Percent FRL 65.57 (19.55) 57.88 (21.83) -0.36 
Percent LEP 12.50 (12.56) 15.02 (20.27) 0.14 
Percent SWD 12.24 (4.26) 13.09 (4.03) 0.21 
Percent White 48.84 (39.56) 47.56 (34.91) -0.03 


Following school-level matching, we compared the baseline mathematics achievement at the 
student level. For each grade level, we compared the fall /-Ready Diagnostic scores of all 
students in the sampled treatment and comparison schools. Within the treatment schools, only 
those students who met the inclusion criteria for using i-Ready Instruction with fidelity were 
included in the sample (e.g., students using /-Ready Instruction for an average of at least 30 
minutes per week for a minimum of 18 weeks). For grades 1 through 5, baseline equivalence 
was achieved at the student level on the fall 2017 mathematics achievement using all students 
in the schools. That is, for these grades, the baseline difference in student achievement had an 
effect size of less than 0.25 (Table 2). This is below the WWC threshold for baseline 
equivalence (WWC, 2017b). Student-level matching was required for kindergarten to create 
samples of treatment and comparison students with baseline mathematics achievement within 
the WWC threshold. 


For kindergarten, student-level baseline equivalence was achieved using propensity score 
matching similar to what was used for the school matching. We used the fall 2017 mathematics 
i-Ready Diagnostic score as our matching variable. We matched one comparison student to 
each treatment student. 


Table 2 provides student-level baseline descriptive statistics on mathematics achievement for 
the study sample used in the impact analyses. The adjusted mean difference between students 
at each grade level was estimated using a mixed model that nested students within schools and 
estimated the difference between the treatment group and the comparison group at the school 
level. This model mirrors the model used to estimate impacts described in the next section. 
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Table 2. Baseline Equivalence Statistics for i-Ready Diagnostic Only (Comparison) and Blended Core Mathematics 
(Treatment) Groups, by Grade. 


Grade Group [exes | Schools | Students ee i-Ready SD Sy eo ol getect 

K i-Ready Only 0.19 21 889 337.98 20.74 -1.84 (3.35) -0.08 
BCM 18 889 336.14 23.00 

1 i-Ready Only 0.20 38 1,987 375.69 23.81 -0.70 (3.00) -0.03 
BCM 25 1,470 374.99 24.87 

2 i-Ready Only 0.15 44 2,347 402.34 24.09 -4.08 (2.68) -0.17 
BCM 24 1,588 398.26 24.92 

3 i-Ready Only 0.17 47 2,221 425.72 26.21 -0.44 (2.95) -0.02 
BCM 24 1,751 425.28 26.52 

4 i-Ready Only 0.15 51 2,628 451.09 26.85 -2.25 (2.86) -0.08 
BCM 26 1,782 448.84 27.62 

5 i-Ready Only 0.24 42 2,167 466.92 31.50 5.16 (4.38) 0.16 
BCM 25 1,796 472.07 32.24 


Notes: BCM = Blended Core Mathematics, ICC = intraclass correlation, SD = standard deviation of i-Ready scores, Adj Mean Diff = 
adjusted mean difference between Blended Core Mathematics and i-Ready Diagnostic only groups, and SE = standard error of the 
adjusted mean difference. 
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Impact Analysis 


Following the selection of baseline equivalent groups, hierarchical linear modeling (HLM) was 
used to address our research question to estimate the impact of Blended Core Mathematics on 
student mathematics achievement. A two-level model was used to account for the clustered 
nature of the data with students nested within schools. Because effect size differences between 
the treatment and comparison on student achievement at baseline fell between 0.05 and 0.25 
standard deviations, baseline mathematics achievement was included in the model as a 
covariate. 


Level 1 of the model was specified as follows: 
Yii = Bo; + Ba(PRE_MATH, — PRE_MATH,)j + ej 


where Yj; is the spring -Ready mathematics diagnostic score for student / in school j. Bo; is the 
adjusted mean outcome for students in school /. 81; is the adjusted difference in outcome due to 
the student’s pretest score in mathematics (cluster mean centered). e; is the random error in the 
achievement outcome associated with student / in school / not accounted for in the model. 


Level 2 of the model was specified as follows: 
Boj = Yoo + Yor( GROUP); + Yo2(PRE_MATH.;— PRE_MATH..); + 2yx(SCHOOL)+ Ug 


where Yoo is the adjusted comparison group grand mean of the outcome, yo is the adjusted mean 
difference in the outcome between school study groups, and GROUP is an indicator variable coded 
as 1 for schools in the Blended Core Mathematics group and 0 for schools in the i-Ready Diagnostic 
Only comparison group. Yoz is the regression slope of the school-level pretests in mathematics 
(grand mean centered) to explain additional between-school variance not explained in level 1 of the 
model. y« is a vector of school-level demographic dummy variables added to increase statistical 
precision. Ug is the random error in the achievement outcome associated with school /. 


The student-level covariate used in each analysis was: 
e j-Ready Diagnostic mathematics baseline performance 
The school-level covariates included: 


e Group membership 
e Average i-Ready Diagnostic fall score 
e Demographic variables 
- Percent white students 
- Percent Limited English Proficient (LEP) students 
- Percent of students eligible for free and reduced-price lunch (FRL) 
- Percent of students with disabilities (SWD) 


The above variables were selected for inclusion in the model because they are exogenous and 
could reasonably be expected to be related to student achievement. 


To indicate the size of impacts, effect sizes were computed for all comparisons using Hedge’s g 
with an adjustment for small sample sizes (WWC, 2017b). 


An Impact Evaluation of the Blended Core Mathematics Implementation for Elementary Grades. 9 


GF HumMRRO 
Impact Analysis Results 


This section describes the results of the HLM analysis. Full information on the HLM model 
results, including student- and school-level covariate parameters, can be found in Appendix B. 
Table 3 summarizes the impact analysis findings. As shown, the Blended Core Mathematics 
schools were found to perform significantly better (with p-values of less than 0.05) than the /- 
Ready Diagnostic only schools on mathematics achievement as measured by student 
mathematics i-Ready Diagnostic scores for all six grades examined. 


The effect sizes, as measured by Hedge’s g, for the six positive impacts ranged from 0.17 
(grade 3) to 0.36 (grades K and 1). The effect sizes for grades K, 1, 2, 4, and 5 are all 0.20 or 
larger, and four of the six effect sizes were .25 or larger which can be considered large (Lipsey 
et al., 2012) and substantively important (WWC, 2018b). 


The intraclass correlations (ICCs) are also presented in Table 3. The |CCs measure the 
proportion of the variance that is between schools—that is, how much of the variance in 
mathematics i-Ready Diagnostic scores that can be explained by school-level differences. The 
ICCs range from 0.17 (grade 1) to 0.30 (grade K). This suggests the majority of variance is due 
to factors other than school-level differences. 


The adjusted mean differences presented in Table 3 indicate students in the Blended Core 
Mathematics (treatment) schools earned higher scores on the spring mathematics i-Ready 
Diagnostic as compared to students in the -Ready Diagnostic only (comparison) schools. Figure 1 
graphically presents the gains in student mathematics performance for each group between the 
fall of 2017 and spring of 2018. As shown, both groups saw gains in mathematics achievement at 
all grade levels. At all grade levels, those in the Blended Core Mathematics experienced greater 
gains than those in the i-Ready Diagnostic only group. 
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Table 3. Impact Analysis Results for Blended Core Mathematics (Treatment) Schools Compared to i-Ready Diagnostic Only 
(Comparison) Schools for Mathematics Student Achievement at grades K-5. 


: ; : Hedge’s 
ee Group ICC ae Sudens ate) ue cous amen leean p-value | g Effect 


Mean =D) Diff (SE) 


Size 

K i-Ready Only | 0.30 | 21 889 368.89 | 23.79 9.25 (3.72) | 013 0.36 
BCM 18 889 378.13 27.67 

1 i-Ready Only 0.17 38 1,987 404.38 25.94 9.33 (1.62) <.001 0.36 
BCM 25 1,470 413.71 25.99 

2 i-Ready Only 0.18 44 2,347 429.58 27.36 6.88 (1.43) <.001 0.25 
BCM 24 1,588 436.46 27.01 

3 i-Ready Only 0.21 47 2,221 455.92 28.92 5.06 (1.81) .005 0.17 
BCM 24 1,751 460.98 30.25 

4 i-Ready Only 0.23 51 2,628 475.42 30.39 6.33 (1.37) <.001 0.21 
BCM 26 1,782 481.75 31.57 

5 i-Ready Only 0.29 42 2,167 485.96 33.59 8.72 (1.54) <.001 0.26 
BCM 25 1,796 494.67 33.74 


Notes: BCM = Blended Core Mathematics, ICC = intraclass correlation, SD = standard deviation of i-Ready scores, Adj Mean Diff = 
adjusted mean difference between Blended Core Mathematics and i-Ready Diagnostic only groups, and SE = standard error of the 
adjusted mean difference. 
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Figure 1. Changes in Mathematics Achievement Between Fall 2017 and Spring 2018, by Grade. 
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Impact Summary and Discussion 


Our findings suggest participation in Blended Core Mathematics resulted in higher student-level 
achievement in mathematics, as measured by the /-Ready Diagnostic, compared to use of /- 
Ready Diagnostic only. The mean mathematics achievement for the Blended Core Mathematics 
group was statistically significantly higher for all of the grades K-5. Moreover, the effect sizes 
showed additional support that students in Blended Core Mathematics schools benefitted from 
their school’s adoption and implementation of i-Ready Instruction and Ready Mathematics 
curriculum. For four of the six grades, the effect sizes were 0.25 or higher. A standardized effect 
size of this magnitude is considered noteworthy in educational research (Lipsey et al., 2012; 
WWC, 2017b) where it is often difficult to observe large effects. All schools attempt to provide a 
valuable education to their students by implementing quality curriculum and classroom 
assessments. Therefore, all students should expect to see gains in student achievement. It is 
important to note the effects of the Blended Core Mathematics treatment were beyond the mean 
performance of schools that used other curriculum options. 


The study was conducted as a rigorous quasi-experimental design (QED) to meet the standards 
described in the WWC 4.0 standards to achieve a rating of Meets WWC Group Design Standards 
with Reservations. |n addition, because we found statistically significant positive results for all 
grades examined, this study meets the guidelines set forth by ESSA for a Level 2 (or Moderate) 
rating for evidence-based research (U.S. Department of Education, 2016). 


This study was a QED. Schools in both groups were not participants in a research study but 
actual customers and everyday users of educational products, and we relied on implementation 
of Blended Core Mathematics carried out in real-world conditions. Implementation of Blended 
Core Mathematics, therefore, likely varied between schools. We may have found different 
results had the study been conducted under more controlled circumstances. Impacts are 
typically greater for studies that aim for ideal or close to ideal implementation and less for 
studies that examine real-world implementation. However, despite this limitation, we were able 
to find solid impacts for all elementary grade levels examined. 


Finally, our treatment group was compared to an i-Ready Diagnostic only group. It is possible 
that use of i-Ready Diagnostic increases student achievement; however, the design of this study 
did not allow for an estimation of that impact. Further, use of the /-Ready Diagnostic only 
schools as a comparison group may have attenuated the effects of the treatment had that group 
been compared to a “business-as-usual” comparison group. Future studies might seek to 
examine the impact of Blended Core Mathematics or i-Ready Diagnostic Only to a set of a 
comparison schools not implementing any Curriculum Associates products. This would require 
an external achievement measure, potentially state assessments, for use as an achievement 
baseline measure and outcome variable. 


Quality Control Procedures 


We employed various quality control checks throughout the data cleaning, analysis, and 
reporting process. HumRRO, Curriculum Associates, and Century Analytics worked together to 
identify a rigorous methodology based on proper implementation of Blended Core Mathematics, 
the WWC 4.0 standards, and ESSA Level 2 guidelines. 


Rules for identifying treatment and comparison groups were determined upfront through 
collaboration between the three groups. Curriculum Associates provided information on the 
various components of Blended Core Mathematics, and the frequency for which they should be 
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used for solid implementation. They provided /-Ready Diagnostic and Instruction data to allow 
HumRRO and Century Analytics to empirically examine the extent to which these 
recommendations were followed by Blended Core Mathematics schools, and they provided 
information on Ready Mathematics implementation based on their experiences interacting with 
identified schools. These discussions led to a treatment and comparison school criteria that all 
partners were confident in. 


Data analysis work was completed collaboratively by HumRRO and Century Analytics. Century 
Analytics and HumRRO independently conducted matching and HLM analyses for each grade. 
The researchers reviewed results against each other and worked out any discrepancies. All 
data reported in this study were verified by both researchers. 
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Appendix A. 


Ready Blended Core Mathematics Logic Model 


Ready Blended Core Mathematics Logic Model 


Resources Strategies and Activities | Outputs Outcomes Impacts 
=p = ua = 
= Teachers attend = Students actively = Students’ personal = Increased student 
i-Ready Diagnostic professional participate in Ready learning needs are met achievement 
development and utilize Mathematics instruction 
i-Ready Online PD resources to acquire = Students have a = Increased student 
Instruction teaching and product = Students take lessons stronger understanding learning gains 


Ready Mathematics as a 
Core program 


skills 


Teachers use Ready 
Mathematics daily to 
facilitate student-driven 
learning experiences and 
are implementing 
instructional and 
assessment components 
with fidelity 


Teachers receive 
information on students’ 
skills and progress 
through Ready 
assessments, informal 
classroom discourse and 
mathematical practices 


Teachers administer the 
Diagnostic to determine 
students’ learning needs 


Teachers use i-Ready 
Instruction to support 
students in content areas 
outside of whole-class 
instruction 


Teachers use i-Ready and 
Ready Mathematics data 
to inform and deliver 

differentiated instruction 


based on their 
performance on the i- 
Ready diagnostic 


= Teachers regularly 
differentiate instruction 
based on i-Ready reports 
and information 
gathered through Ready 
mathematics tools 


of grade level 
mathematics content 


Students develop habits 
of mind and standards 
of practice to improve 
mathematical reasoning 
and perseverance 
through challenging 
topics 


© Curriculum Associates, LLC. (July, 2019). Blended Core Ready Mathematics Logic Model. North Billerica, MA: Author. Report prepared 
07/10/2019. 
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Table B.1. HLM Results for Blended Core Mathematics Compared to i-Ready Diagnostic 


SE z p-value | 95% Conf. Interval 


Only for Kindergarten. 


| Covariates 


Treatment Group Membership 9.25 3.72 2.48 0.013 1.95 16.54 
a 0.70 0.22 3.19 0.001 0.27 1.14 
School-Level Covariates 
Percent FRL 23.73 14.32 1.66 0.097 -4.34 51.79 
Percent SWD 42.29 45.37 0.93 0.351 -46.63 131.20 
Percent LEP -4.75 9.78 -0.49 0.628 -23.92 14.43 
Percent White 0.19 0.08 2.36 0.019 0.03 0.35 
Fall Mathematics i-Ready 
School-Level Grand Mean 0.71 0.02 34.96 <0.001 0.67 0.75 
Centered 
Intercept | 342.97 10.88 31.53 <0.001 321.65 364.29 


Notes: FRL = free or reduced lunch, SWD = students with disabilities, LEP = limited English proficient, 
Coef. = coefficient, SE = standard error of the coefficient, z = standardized score 


Table B.2. HLM Results for Blended Core Mathematics Compared to i-Ready Diagnostic 


Only for Grade 1. 
Covariates (ore) -) aS) z | p-value | 95% Conf. Interval 
Treatment Group Membership 9.33 162) 577 <0.001 6.16 12.50 
ee ee 0.09 8.01  <0.001 054 0.90 
School-Level Covariates 
Percent FRL -0.65 5.30 -0.12 0.903 -11.04 9.75 
Percent SWD -6.88 19.80 -0.35 0.728 -45.68 31.92 
Percent LEP -0.80 4.23 -0.19 0.850 -9.09 7.48 
Percent White 0.04 0.04 1.22 0.222 -0.03 0.12 
Fall Mathematics i-Ready 
School-Level Grand Mean 0.82 0.01 65.68 <0.001 0.79 0.84 
Centered 
Intercept 402.13 4.45 90.43 <0.001 393.41 410.85 
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Table B.3. HLM Results for Blended Core Mathematics Compared to i-Ready Diagnostic 


Only for Grade 2. 
ONE oe alue Shey {in Ofe 
Treatment Group Membership 6.88 1.43 4.81 <0.001 4.07 9.68 
Be tr tees He 0.76 0.08 9.45 <0.001 0.60 0.92 
School-Level Covariates 
Percent FRL -4.47 5.06 -0.88 0.377 -14.38 5.45 
Percent SWD -14.55 17.62 -0.83 0.409 -49.08 19.99 
Percent LEP 6.08 3.76 1.62 0.106 -1.29 13.46 
Percent White 0.12 0.03 3.94 <0.001 0.06 0.18 
Fall Mathematics i-Ready 
School-Level Grand Mean 0.88 0.01 82.01 <0.001 0.86 0.90 
Centered 
Intercept 424.66 4.13 102.74 <0.001 416.56 432.77 


Table B.4. HLM Results for Blended Core Mathematics Compared to i-Ready Diagnostic 


Only for Grade 3. 
ova oe 95% Co 
Treatment Group Membership 5.06 1.81 2.79 0.005 1.51 8.61 
oe; ee 0.10 10.02 <0,001 0.77 1.14 
School-Level Covariates 
Percent FRL -4.86 6.52 -0.75 0.456 -17.63 7.91 
Percent SWD -2.10 22.42 -0.09 0.925 -46.03 41.84 
Percent LEP 16.39 4.98 3.29 0.001 6.62 26.15 
Percent White 0.05 0.04 1.31 0.191 -0.03 0.13 
Fall Mathematics i-Ready 
School-Level Grand Mean 0.90 0.01 86.98 <0.001 0.88 0.92 
Centered 
Intercept 449.46 5.24 85.71 <0.001 439.18 459.74 
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Table B.5. HLM Results for Blended Core Mathematics Compared to i-Ready Diagnostic 


Only for Grade 4. 
ere ele a E 95% e a 
Treatment Group Membership 6.33 1.37 4.61 <0.001 3.64 9.03 
Be tr tees He 1.10 0.06 1869 <0.001 0.99 1.22 
School-Level Covariates 
Percent FRL -2.72 4.50 -0.60 0.545 -11.53 6.09 
Percent SWD -38.88 17.29 -2.25 0.025 -72.76 -4.99 
Percent LEP 12.77 4.22 3.02 0.002 4.50 21.05 
Percent White 0.07 0.03 2.52 0.012 0.02 0.13 
Fall Mathematics i-Ready 
School-Level Grand Mean 0.95 0.01 102.49 <0.001 0.93 0.96 
Centered 
Intercept 471.29 3.77 125.09 <0.001 463.91 478.67 


Table B.6. HLM Results for Blended Core Mathematics Compared to i-Ready Diagnostic 


Only for Grade 5. 
OVa el: Shey (in OFe 
Treatment Group Membership 8.72 1.54 5.65 <0.001 5.69 11.75 
fis reste ae nts 1.03 0.05 19.19 <0.001 0.93 1.14 
School-Level Covariates 
Percent FRL -1.18 5.23 -0.23 0.821 -11.43 9.07 
Percent SWD -4.68 21.47 -0.22 0.828 -46.76 37.40 
Percent LEP 4.38 4.93 0.89 0.374 -5.28 14.04 
Percent White 0.02 0.03 0.49 0.628 -0.05 0.08 
Fall Mathematics i-Ready 
School-Level Grand Mean 0.91 0.01 106.15 <0.001 0.89 0.92 
Centered 
Intercept 480.74 4.52 106.35 <0.001 471.88 489.60 
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